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Abstract 


This  paper  reflects  a  successful  effort  to  apply  commercial  off-the-shelf  (COTS)-based 
engineering  principles  to  a  software  acquisition  by  the  Financial  and  Business  Services 
(FABS)  and  Information  Technology  (IT)  departments  at  the  Software  Engineering  Institute. 
The  team  responsible  for  the  execution  of  the  project  was  guided  by  the  principles  taught  in 
the  “COTS-Based  Systems  for  Program  Managers”  and  “COTS  Software  Evaluation  for 
Practitioners”  training  programs  conducted  by  the  COTS-Based  Systems  Initiative  at  the 
Software  Engineering  Institute.  Some  of  the  major  expectations  set  and  realized  included 
precise  comprehension  of  requirements  and  preferences,  ability  to  identify  weak  links  in  the 
proposed  solutions,  support  for  the  “buy  versus  build”  decision  and  the  product 
recommendation,  the  promise  of  a  shorter  implementation  phase,  and  brimming  confidence 
based  on  a  well-informed  project  approach. 
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1  Introduction 


In  late  1999,  Carnegie  Mellon  University  (CMU)  had  implemented  the  Oracle  Enterprise 
Resource  Planning  (ERP)  System,  a  suite  of  business  applications.  The  ERP  system  replaced  the 
then  existing  transaction  processing  systems,  with  the  promise  of  extending  standardization — 
across  the  global  organization — of  the  way  financial  transactions  would  be  recorded,  managed, 
and  maintained.  The  Software  Engineering  Institute  (SEI)  was  among  several  constituents  of 
CMU  that  were  to  be  served  with  this  system  through  a  central  Information  Technology  (IT) 
services  deployment.  The  system  afforded  several  extraordinary  capabilities  for  automating 
bookkeeping  and  consolidation  functions.  But  it  had  certain  shortcomings:  one  was  an  inability 
to  maintain  budgets  in  a  manner  that  would  serve  the  business  goals  of  the  SEI.  This  was  a  major 
concern.  The  situation  warranted  the  continued  use  of  the  faltering  budget  system  that  had  been 
built  internally  several  years  prior  (1993).  The  system  was  faltering  because  it  needed  to  be 
modified  substantially  to  accommodate  several  new  needs  created  by  the  advent  of  the  Oracle 
ERP  system. 

We  found  that  the  dream  of  an  ERP  system  eliminating  all  the  problems  would  elude  us.  We  were 
trying  to  establish  dependencies  between  two  systems  that  had  no  capability  built  in  to  support 
such  an  effort.  Thus  arose  the  need  to  find  a  suitable  solution  to  a  serious  problem.  We  saw  two 
choices: 

1.  Change  the  business  process  to  suit  the  ERP  system. 

2.  Appropriately  change  the  existing  in-house  system. 

We  did  not  consider  the  first  option,  since  it  had  potential  to  affect  a  wide  section  of  the  CMU 
business  community,  and  would  have  required  elaborate  business  considerations  specific  to  the 
various  entities  within  CMU  and  increased  time  to  deploy  a  solution  in  a  project  of  such  scale  and 
complexity. 

We  chose  the  second  option  and  ran  into  the  wall,  since  reality  does  not  allow  the  molding  of  one 
product  to  fit  another’s  needs  without  adverse  consequences.  Challenges  were  posed;  we  would 
meet  the  challenge  and  attempt  to  move  on.  The  monster  would  raise  its  head  and  exhibit  a 
different  behavior  and  be  tamed  again  until  all  the  troubles  surfaced  at  once,  taking  us  to  a  point 
of  no  return.  We  found  the  main  ill  effects  of  this  phenomenon  to  be  the  tremendous  effort 
directed  at  keeping  the  business  process  from  failing.  Declining  productivity,  user  frustration, 
delayed  decisions,  quick  workarounds,  and  questionable  numbers  were  some  effects  that  we 
experienced.  At  this  juncture,  we  reached  a  decision  to  scour  the  marketplace  in  an  effort  to  find  a 
commercial  off-the-shelf  (COTS)  solution.  A  team  of  five  members — a  combination  of  technical 
and  business  domain  experts— were  called  upon  by  management  to  handle  the  task  of  steering  the 
rocky  boat  to  calmer  seas.  This  paper  attempts  to  shed  light  on  the  process  with  the  hope  that  it 
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can  serve  as  a  reference  for  the  readers  who  expect  to  embark  on  projects  involving  COTS 
acquisitions. 
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2  Preparing  for  the  Process 


Finding  a  COTS  product  proved  problematic.  This  was  partly  because  the  peculiarities  of  our 
ITRDC  (Federally  Funded  Research  and  Development  Center)/Higher  Education  financial 
process  were  not  entirely  addressed  by  the  marketplace.  Therefore,  the  Chief  Financial  Officer 
was  inspired  to  request  that  the  COTS  Program  specialists  provide  guidance  on  the  project,  with 
intent  to  procure  an  off-the-shelf  solution.  We  cannot  but  be  thankful  to  him  for  that  decision. 
Training  in  COTS  software  acquisition  techniques  soon  followed.  The  techniques  learned  were 
applied  in  the  product  evaluation  process.  The  COTS  Program  specialist(s)  participated  in  each 
meeting.  This  greatly  influenced  the  coining  of  the  title  to  this  paper’s  series.  The  application  of 
these  techniques  is  discussed  at  length  in  this  paper.  Each  sub-section  below  maps  to  the  process 
recommended  in  the  COTS  training,  and  was  of  course  tailored  to  suit  our  situation. 


Figure  1:  Diagrammatic  Representation  of  the  Product  Selection  Process 


2.1  Problem  Statement  Definition 

It  was  very  important  that  we  understand  the  problems  with  the  existing  system  so  we  could 
determine  the  needs  to  be  satisfied  by  the  new  system.  This  knowledge  later  helped  us  spot 
weaknesses  in  the  proposals  resulting  from  our  quest.  After  evaluating  all  options  available  in- 
house  and  on  campus  we  concluded  that  much  was  left  to  be  desired  towards  fulfilling  the 
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organization’s  needs.  For  example,  we  needed  the  capacity  to  budget  by  dollars  and  full  time 
equivalent  (FTE),  budget  people  across  projects  and  vice  versa  with  an  automatic  consolidation 
capability,  accommodate  multiple  accounting  structures  within  one  application,  and  reduce  the 
maintenance  overhead.  In  attempting  to  derive  these  capabilities,  we  were  conscious  of  ensuring 
that  problems  (for  example,  the  inability  to  integrate  with  other  systems  critical  to  the 
environment,  inability  to  make  modifications  driven  by  change),  which  were  agonizing  attributes 
of  the  present  system,  would  also  need  to  be  mitigated. 

Given  the  above  situation  we  reached  a  decision  to  go  out  to  the  marketplace  to  find  a  solution 
that  would  not  only  address  the  deficiencies  that  we  were  encountering  with  the  available 
SEI/CMU  systems,  but  would  also  provide  some  absolutely  essential  capabilities  that  would 
simplify  the  process  of  budgeting  and  reporting. 


2.2  Defining  Stakeholders 

COTS  product  acquisition  techniques  suggest  that  the  right  mix  of  stakeholders  should  be 
involved  on  the  project.  It  is  important  to  identify  the  appropriate  stakeholders,  since  they  will 
bring  considerable  knowledge  regarding  the  usefulness  of  various  aspects  of  the  product.  To  aid 
in  identifying  the  right  combination  of  stakeholders,  we  considered  the  following  questions. 

Who  would  derive  value  from  the  system? 

In  our  case  the  business  managers  of  various  research  initiatives  and  the  financial  groups  that 
were  supporting  these  business/program  groups  were  to  derive  the  maximum  value  by  way  of 
being  able  to  record,  maintain,  manage,  and  report  on  the  budgets  and  the  related  actual 
expenditure. 

Who  would  sponsor  the  project? 

The  onus  was  on  the  Financial  and  Business  Services  Management  Group  to  provide  this  key 
management  function,  so  they  would  be  driving  the  project. 

Who  would  grant  post-implementation  support? 

Representatives  of  the  Information  Technology  organization  that  would  house  and  administer  the 
new  system  were  brought  on  board,  to  contribute  to  the  understanding  of  the  new  system’s  impact 
on  the  IT  Enterprise  Level  Infrastructure. 

Who  owns  the  systems  that  the  new  system  will  depend  on  or  may  have  to  guarantee  services  to? 

Functional  and  IT  organizations  representing  the  various  systems  that  were  to  integrate  with  the 
new  system  were  included.  This  would  foster  a  collective  understanding  of  the  changes  in  the 
enterprise  architecture  that  the  new  system  would  cause  or  require. 
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Who  would  support  the  contractual  and  procurement  aspects  of  the  acquisition  process? 

We  brought  on  board  personnel  from  the  Purchasing  Offices  on  campus  and  within  the  SEI  to 
ensure  an  understanding  of  the  financial  and  legal  obligations  to  which  we  would  be  committing 
ourselves.  Their  oversight  would  also  help  avoid  licensing  restrictions  that  might  hinder 
appropriate  use  or  bear  negative  cost  impact. 

Representatives  from  these  areas  with  necessary  functional  and  technical  expertise  were 
determined  to  be  the  stakeholders.  Defining  the  group  in  this  manner  would  ensure  that  we  could 
draw  upon  their  expertise  to  guide  the  project  to  its  desired  destiny.  Their  participation  — to  the 
ideal  degree — ^was  necessitated  by  the  fact  that  multiple  systems  would  be  involved  in  the  final 
integration. 

The  team  members  who  had  the  task  of  selecting  stakeholders  understood  the  organizational 
structure  to  which  the  various  stakeholders  belonged.  This  enabled  them  to  identify  the  desired 
players  based  on  their  usefulness  in  rendering  value  and  success  to  the  project. 

We  were  very  fortunate  in  having  stakeholders  who  were  very  cooperative  during  the  entire 
process  and  were  always  willing  to  contribute  their  best,  through  the  sharing  of  their  valuable 
knowledge  and  expertise,  which  we  believe  was  very  critical  to  the  success  of  the  evaluation 
process. 


2.3  Definition  of  Requirements  and  Preferences 

COTS  solutions  must  be  evaluated  for  their  ability  to  satisfy  the  requirements  that  initiated  their 
quest.  Acquiring  thorough  knowledge  of  requirements  is  highly  essential  as  well  as  time 
consuming.  Typically,  adequate  time  is  not  spent  in  this  regard.  It  is  one  of  the  areas  one  must 
revisit  at  the  first  signs  of  major  usability  inefficiencies  in  the  deployed  system.  There  are  two 
approaches  to  this  phase  within  COTS  based  implementations: 

•  Build  or  change  functional/technical  environment  around  that  of  the  COTS  product  chosen. 

•  Find  the  COTS  product  that  is  most  suitable  to  the  functional/technical  environment 
prevalent. 

The  chance  of  finding  a  product  that  satisfies  all  requirements  is  remote.  At  best  a  COTS  product 
might  satisfy  several  requirements,  but  rarely  all.  In  such  cases  tradeolfs  may  be  essential  or 
changes  to  the  functional/technical  environment  may  be  necessary.  Requirements  that  still  need  to 
be  satisfied  are  addressed  through  additional  COTS  components  or  by  custom  development.  In 
either  case  it  is  important  that  we  know  the  requirements.  Before  we  proceed  any  further,  let  us 
understand  the  term  “requirement.” 
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A  requirement  is  a  functional  or  technical  need  that  must  be  satisfied  by  the  chartered  system 
within  the  given  operating  environment.  Failure  to  satisfy  any  of  the  requirements  bears  a  highly 
adverse  and  unjustifiable  functional  or  technical  disability  in  fulfilling  the  main  objectives  of  the 
implementing  organization.  Obviously  the  people  who  should  be  involved  in  the  definition  of 
requirements  are  those  who  use  it  and  maintain  it;  their  help  makes  it  easier  to  identify  the  broad 
business  and  technical  needs  that  must  be  satisfied  by  the  system.  In  our  project,  we  assigned 
people  to  gather  these  needs  and  document  the  specifics  under  each  category.  Within  a  couple  of 
brainstorming  sessions,  a  list  of  11  requirements  came  to  light  and  a  partial  list  is  reproduced  in 
Table  1  in  Section  4.  This  process  brought  about  an  understanding  of  the  current  environment  in 
light  of  these  requirements,  the  needs  for  the  future,  and  the  evolutionary  change  that  must  be 
factored  within  the  scope  of  each  of  these  requirements. 

The  approach  we  took  was  as  follows: 

1.  Identify  the  external  systems  involved  namely,  Oracle  ERP  system  and  Human  Resource 
Information  (HRIS)  system. 

2.  Identify  the  system  environment  in  both  the  above  cases. 

3.  Identify  the  system  environment  into  which  the  new  system  would  be  introduced. 

4.  Identify  the  business  processes  that  are  mandatory. 

5.  Identify  the  areas  causing  functional  frustration. 

6.  Identify  the  persons  that  have  a  good  understanding  of  the  various  aspects  and  pair  them  up 
to  deliver  the  requirements  specifications. 

7.  Identify  the  direction  of  the  organization  in  relation  to  the  marketplace  and  the  technology. 

8.  Brainstorm  to  segregate  the  list  into  requirements  and  preferences. 

9.  Draw  up  a  functional  specifications  document. 

It  is  not  an  uncommon  industry  practice  to  select  the  vendor  and  then  have  the  requirements 
drafted.  However,  this  approach  bears  the  potential  of  opening  the  doors  for  modification  of  a 
vendor’s  system  to  fit  the  requirements.  In  the  event  this  were  not  feasible,  processes  would  need 
to  be  changed  or  complex  customizations  would  have  to  be  made,  in  both  cases  causing  fresh 
management  issues  (namely,  resistance  to  changed  processes  or  management  of  increased 
complexity).  Unless  the  benefits  of  this  approach  far  outweigh  the  risks  and  adversities  they  bring 
on,  it  is  unwise  to  tread  this  path.  As  much  as  vendors  know  their  wares,  the  customers  know  their 
needs.  It  is  important  in  the  COTS  acquisition  world  that  the  customer  spells  out  the  needs  in  no 
uncertain  terms.  The  right  mix  of  stakeholders  discussed  in  the  earlier  section  would  help  to  a 
great  extent  in  supporting  the  requirements-gathering  and  the  impact-analysis  phases  of  the 
project. 

We  believe  the  approach  we  took  has  paid  in  the  final  analysis.  And  what  is  this  analysis?  It  is  the 
fact  that  we  were  able  to  use  the  characteristics  represented  by  the  various  requirements  to 
develop  the  criteria  and  the  capability  statements  to  assist  in  the  evaluation  process.  We  are  glad 
that  we  achieved  this  clarity  up  front.  We  feel  we  have  prevented  the  ill  effects  of  making  haste 
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that  may  have  occurred  had  the  vendors  been  onsite  and  the  meters  been  ticking.  Given  such 
constraints  of  cost  and  time,  we  could  have  exposed  ourselves  to  a  distorted  focus  during  a 
critical  phase  of  the  effort.  Because  of  our  approach,  we  expect  our  project  costs  to  be  relatively 
stable.  Equally  important  is  the  fact  that  having  defined  requirements  and  preferences  would 
make  it  easier  to  seek  what  we  need  in  the  proposed  solutions.  Resulting  from  the  approach  was 
the  Functional  Specifications  document,  which  served  as  an  attachment  to  the  Request  for 
Proposal  (RFP)  and  a  reference  point  for  evaluations. 

Our  approach  has  paid  off  by  enabling  us  to 

•  establish  the  criteria  that  would  be  used  to  evaluate  the  proposals 

•  reason  out  the  ability  of  the  new  system  to  fit  the  technical  environment 

•  communicate  to  the  prospective  vendors  that  only  certified  and  tested  interfaces  are  needed  to 
integrate  the  dependent  systems 

•  communicate  to  the  prospective  vendors  the  data  structures  and  the  dependencies  between 
systems 

•  discuss  with  the  prospective  vendors  the  business  rules  and  the  complexities  at  great  length 

•  stick  to  one  stance  at  all  times  with  the  prospective  vendors 

•  communicate  to  the  prospective  vendors  the  business  value  that  we  anticipated  from  the 
implementation 


2.4  Definition  of  Expectations  from  the  New  System 

COTS  acquisition  techniques  suggest  the  definition  of  expectations  and  the  methods  to  identify 
capabilities  of  the  proposed  solutions  to  satisfy  the  requirements  desired.  To  this  end,  the 
expectations  and  the  means  to  assess  the  capabilities  were  drafted  to  assist  in  the  evaluation 
process.  They  represented  the  characteristics  and  features  desired  of  the  proposed  systems.  They 
were  grouped  into  appropriate  categories  namely.  System  Environment,  Operating  Environment, 
Functionality,  Vendor  Attributes,  COTS  Characteristics,  and  so  on  as  partially  illustrated  in  Table 
2  in  Section  5.4. 


2.5  Definition  of  Expectations  About  Vendor 

In  an  attempt  to  make  sure  we  understood  whom  we  were  dealing  with  we  had  prepared  a  vendor 
survey.  Parts  of  the  survey  sought  information  about  the  vendor  including  its  Dunn  and  Bradstreet 
report,  financial  state  of  affairs,  client  list,  and  so  on.  We  did  this  well  in  advance  of  product 
selection.  This  helped  us  avoid  the  risky  position  of  having  to  make  compromises  to 
accommodate  a  faulty  vendor  selection. 
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2.6  Definition  of  the  Grading  Scaie 

The  COTS  acquisition  process  advises  the  use  of  some  reasonable  method  of  quantifying  the 
inferences  drawn  about  a  product’s  ability  to  satisfy  expectations.  While  there  exist  automated 
packages  for  performing  this  task,  we  chose  to  define  a  scale  (illustrated  in  Table  3  in  Section  5.4) 
for  the  purpose.  The  scale  was  to  measure  the  existence  of  functionality  or  lack  of  it,  ability  to 
modify  or  lack  of  it,  ease  of  modification  or  lack  of  it,  compatibility  of  the  system  with  the 
current  technical  environment,  perceivable  cost  impact,  and  so  on. 
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3  The  Search  for  a  Solution 


The  process  discussed  in  this  section  is  important  and  critical  to  the  evaluation.  It  helps  avoid 
dealings  with  novice  vendors  who  are  exploring  avenues  to  enter  the  field.  By  skipping  this 
process,  one  may  end  up  spending  substantial  amounts  of  time  and  resources  over  a  meaningless 
proposal — and  probably  be  wedded  to  it  as  well. 


3.1  Representation  at  Key  IT  Conferences 

It  was  important  to  participate  in  the  right  conferences  to  get  an  understanding  of  the  latest 
developments  in  the  market.  This  provided  an  opportunity  to  network  with  vendors  and  with 
representatives  of  other  organizations  battling  setbacks  similar  to  those  discussed  in  this  paper. 
Such  gatherings  also  provided  the  chance  to  draw  the  attention  of  the  ERP  vendors  to  the 
shortcomings  of  their  products. 


3.2  Shortlisting  Probable  Vendors  (COTS  and  Custom  Solution 
Providers) 

Researching  and  creating  a  list  of  probable  vendors  was  a  difficult  task.  Since  “birds  of  a  feather 
flock  together,”  conferences  hosted  by  the  ERP  system  vendor,  Oracle  Corporation,  were  targeted 
for  our  quest.  There  were  vendors  aplenty  who  were  in  some  manner  affiliated  and/or  committed 
to  Oracle’s  business  and  technical  strategy  and  dedicated  to  the  financial  software  arena.  We 
began  talking  with  them,  and  soon  we  were  known  as  a  potential  client  with  a  problem  needing  a 
solution;  before  long  we  went  into  the  serious  business  of  talking  in  detail  with  a  large  number  of 
vendors.  Our  defined  requirements  and  related  documents  proved  useful  at  these  discussions;  this 
form  of  preparedness  reflected  sufficient  seriousness  to  attract  mostly  capable  vendors.  The 
degree  of  reciprocal  seriousness  helped  in  shortlisting  the  vendors. 


3.3  Request  for  Proposal  (RFP) 

To  establish  the  vendors’  familiarity  with  the  business  context  and  their  competency  to  undertake 
the  task,  a  series  of  teleconferences,  site  visits,  and  demonstrations  soon  followed.  An  RFP  was 
released  to  the  vendors  with  intent  to 

•  obtain  the  best  possible  bid 
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•  ensure  an  honest  exchange  in  the  marketplace  and  provide  transparency  to  the  vendors 

•  ensure  compliance  with  the  Department  of  Defense  (DoD)'  guidelines  for  IT  acquisitions 


3.4  On-Site  Pre-Bid  Conference 

The  vendors  were  invited  to  attend  a  mandatory  day-long  on-site  meeting.  The  meeting  was 
divided  into  two  sessions.  It  was  held  to  deal  with  the  technical  and  the  functional  aspects  of  the 
project.  This  served  as  a  platform  to  communicate  and  clarify  the  requirements.  Business  domain 
experts  and  technical  experts  from  the  SEI  and  CMU  were  on  hand  for  this  purpose.  The  meeting 
was  followed  by  a  post-conference  question-and-answer  exchange  via  email,  which  was 
circulated  to  all  regardless  of  the  initiator  of  the  questions. 


3.5  Proposal  Receipt  and  Opening 

After  allowing  substantial  time  for  a  response,  we  received  proposals  from  7  of  the  II  vendors 
who  attended  the  meeting.  Three  of  them  dropped  out  for  reasons  unknown.  One  vendor  decided 
to  refrain  from  participation  since  its  product  was  not  aligned  to  the  business  process  discussed; 
the  fact  that  we  did  not  have  to  deal  with  the  evaluation  of  an  inappropriate  product  was  a  notable 
advantage  resulting  from  our  approach.  To  maintain  the  sanctity  of  the  proposal-opening  process, 
all  stakeholders  were  invited  to  witness  the  event. 


Being  a  Defense  FFRDC,  we  are  guided  by  contractual.  Office  of  Management  and  Budget  (0MB), 
Federal  Acquisition  Regulations,  Defense  Finance  and  Accounting  Service,  Defense  Contract  Audit 
Agency,  and  other  relevant  guidelines. 
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4  Knowledge  of  the  Need 


The  COTS  process  advises  an  absolute  understanding  of  the  needs,  refined  by  appropriate  give- 
and-take  efforts  between  the  various  system  owners  and  the  user  groups.  On  our  project,  this 
understanding  was  encapsulated  in  the  functional  specifications  document  discussed  earlier.  A 
questionnaire  was  used  to  elicit  the  users’  thoughts  on  the  essentials  of  a  planning,  forecasting, 
and  budgeting  solution  that  would  help  them  manage  their  programs  better.  Brainstorming 
sessions  with  key  players  were  held.  The  objectives  of  these  sessions  were  also  to  establish 
critical  changes  related  to  the 

•  way  business  is  being  done 

•  way  information  is  gathered,  processed,  and  analyzed 

•  business  drivers 

•  key  performance  indicators 


Table  1:  Sample  Requirements 


Requirements 

Description 

Data  extracts  from 
General 

Ledger/Grants 
Management  System 
and  Human  Resource 
Information  System 
(HRIS) 

The  system  should  extract  data  through  appropriate 
use  of  interfaces  from  the  Oracle  ERP  and  HRIS 
systems 

Flexible  Reporting 

Should  be  capable  of  providing  highly  flexible 
reports  run  for  different  user-selected  criteria  and 
presented  in  multiple  formats  as  reflected  in  the 
sample  reports  provided 

Burdening 

Should  be  capable  of  complex  conditional 
transformations  and  validation  of  burden  structure 
and  negotiated  rates  for  the  appropriate  funding 
type,  expenditure,  task 

Client  Server 
Architecture 

Must  be  based  on  a  client  server  architecture 

Forecasting 

Must  be  capable  of  extracting  the  actual 
expenditures  to  support  forecasting 

Accounting 

Structures 

Must  be  capable  of  supporting  multiple  accounting 
structures 
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The  consolidated  comments  drove  the  effort  to  freeze  the  scope.  Table  1  above  reflects  the 
encapsulation  of  part  of  our  understanding..  Caution  with  respect  to  “scope  creep”  and  awareness 
of  its  dangers  are  strongly  advised.  No  doubt  requirements  caused  by  changes  to  the  environment 
with  potential  to  derail  the  application,  if  not  already  factored  in,  would  have  to  be  prioritized  for 
inclusion  into  the  scope.  For  example,  at  the  time  of  writing  this  paper,  there  came  into  being  a 
plan  to  upgrade  the  ERP  system  (for  the  second  time  in  a  year).  The  selected  vendor  has  already 
been  cautioned  about  the  upgrade  and  is  expected  to  factor  its  impact  into  the  implementation 
strategy.  It  is  interesting  to  note  that  the  vendor  has  substantial  experience  in  the  type  of  work 
related  to  the  upgrade.  It  makes  it  that  much  easier  to  plan  ahead.  So  the  process  of  educating 
oneself  and  the  vendor  on  matters  affecting  the  enterprise-level  infrastructure  appears  to  be  a 
continuous  one,  expected  to  bear  fruit  at  the  appropriate  time.  This  approach  has  resulted  in  a 
common  understanding  of  the  project  goals  across  the  lifetime  of  the  project.  It  has  aided  in  the 
assessment  of  any  need  for  process  changes.  It  has  also  assisted  in  evaluating  the  need  to 
tailor/modify  the  application  or  modify  our  infrastructure.  Knowing  the  needs  helped  ascertain 
their  satisfaction  in  the  proposals  received,  thereby  helping  us  to  determine  if  we  could  buy  the 
product  versus  building  a  system,  which  is  essentially  the  buy  versus  build  decision  process. 
Evaluation  of  COTS  solutions  preceded  the  evaluation  of  custom  solution  proposals  for  obvious 
reasons — ^time  to  market,  reduced  risk,  and  so  on. 

With  intent  to  understand  our  needs  and  their  order  of  importance,  we  asked  ourselves  the 
questions  listed  below.  They  helped  derive  the  knowledge  that  we  needed  to  continue  our  efforts. 

•  What  is  the  time  to  market? 

•  Do  we  need  a  solution  tailored  to  fit  the  processes  and  external  components  exactly?  If  yes, 
do  we  have  the  resources  to  do  it? 

•  Have  we  an  idea  of  the  post-implementation  need  to  stay  abreast  with  the  developments  and 
evolution  in  the  external  system’s  (Oracle  ERP  system  in  particular)  product  life  cycle  if  we 
were  to  develop  an  in-house  system? 

•  Have  we  an  idea  of  what  amount  of  commitment  in  terms  of  money  and  manpower  is  needed 
to  stay  abreast? 

•  Are  the  benefits  that  we  expect  to  reap  more  than  the  costs  of  attempting  the  solution? 

•  If  we  do  not  adopt  the  idea  of  a  customized  solution,  what  are  our  options  and  how  do  we 
find  them? 

•  What  is  our  level  of  preparedness  to  deal  with  the  bitter  truth  that  there  may  truly  be  nothing 
available  in  the  market  and  we  need  to  go  custom? 

•  If  COTS  products  are  offered  as  a  solution,  how  do  we  know  that  the  solution  will  satisfy  our 
needs  in  their  entirety?  How  do  we  evaluate  the  products  within  the  scope  of  the  functional 
and  technical  environment  prevalent  within  the  organization? 

•  Can  the  product  be  evaluated  reasonably  well  with  the  help  of  a  demonstration? 

•  Would  the  vendors’  demonstration — usually  known  to  represent  a  “right  fit” — hold  true  at  all 
points  of  the  project  life  cycle  and  thereafter? 

•  Would  a  product  evaluation  be  useful? 
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•  Do  we  need  to  be  trained  to  undertake  a  product  evaluation? 

•  Are  we  prepared  to  sink  money  into  the  training  and  discard  if  the  product  is  not  suitable? 

•  If  offered  several  different  solutions,  are  we  prepared  to  evaluate  all  with  appropriate 
training? 

This  is  not  an  exhaustive  list  of  the  questions.  There  were  several  others  that  we  considered.  All 
the  questions  considered  helped  us  understand  the  reality  of  the  situation  that  we  were  faced  with 
and  the  decisions  we  had  to  make.  We  needed  additional  experts  to  guide  us  in  the  process.  And 
fortunately,  we  had  the  COTS  program  specialists  to  guide  us.  They  required  that  we  render  some 
specific  meaning  to  our  effort  by  developing  the  scenarios  and  the  expectations  from  the  solutions 
that  would  satisfy  the  final  goal.  In  the  process,  we  managed  to  prepare  several  artifacts  that 
helped  us  understand  the  scope  of  the  project  and  the  tasks  that  would  have  to  be  performed  to 
achieve  its  goals.  This  formalization  also  satisfied  the  responsibility  of  the  organization  to 
exercise  due  diligence  in  matters  of  investment.  The  artifacts  were  to  form  the  final  basis  of  a 
common  understanding  of  what  we  needed  to  do  our  jobs  better. 
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5  Proposal  Evaluation 


The  effectiveness  of  the  knowledge-gathering  phase  has  a  tremendous  impact  on  the  proposal 
evaluation.  It  provides  the  necessary  guidance  to  look  for  the  appropriate  system  attributes  as 
reflected  through  the  proposals.  Prompting  structured  responses  to  the  request  for  proposal  made 
evaluation  easier.  This  was  accomplished  through  our  structuring  of  the  RFP,  the  functional 
specifications,  and  vendor  survey.  Awareness  of  the  difficulty  of  the  proposal  evaluation  process 
dawned  very  early  in  the  project’s  life  cycle.  The  evaluation  team  determined  the  quality  of  the 
solutions  proposed  and  their  ability  to  mitigate  known  problems. 

In  late  December  2001  (note:  the  project  was  stalled  for  a  period  of  seven  months  due  to  the  ERP 
system  upgrade),  an  RFP  was  released.  The  wait  was  warranted  by  the  absence  of  key  resources 
and  the  intent  to  implement  in  an  upgraded  environment. 


5.1  Proposal  Evaluation  Criteria 

The  proposal  evaluation  criteria  were  made  known  to  the  vendors  through  the  RFP;  but  not  their 
order  of  importance,  lest  proposals  be  fabricated  to  win  the  contract.  The  statement  by  no  means 
intends  to. inflict  disrespect  to  the  vendors.  It  instead  serves  as  insulation  against 
misrepresentation,  while  retaining  the  ability  to  attract  good  proposals.  The  criteria  defined  for 
evaluation  included 

1 .  overhead  processing  functionality 

2.  time  periods  handling 

3.  ability  to  process  business  in  the  pipeline 

4.  forecasting 

5.  revenue  budgeting 

6.  effort  translation  to  report  full  time  equivalent 

7.  new~hire  budgeting 

8.  system  requirements 

9.  client  server  architecture 

10.  vendor  attributes 

1 1 .  COTS  product  characteristics 

12.  commercial  terms 
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5.2  Functional  Specifications  Document 

The  comprehensive  Functional  Specifications  document  referred  to  earlier  spoke  of,  among  other 
things,  the  situation,  the  problem,  the  environment  and  the  goal. 


5.3  Vendor  Survey 

A  vendor  survey,  designed  to  evaluate  the  usefulness  to  the  vendor  of  the  RFP  and  related 
documents,  helped  us  to  gather  feedback  for  assessing  the  need  for  improvement  of  future  RFPs. 
The  survey  also  served  to  provide  information  about  the  vendors  themselves. 


5.4  Process  Definition 

The  project  team  sought  the  guidance  of  the  COTS  acquisition  program  specialists  for  insights 
into  the  method  for  defining  the  criteria.  This  led  to  brainstorming  sessions  that  helped  crystallize 
the  evaluation  criteria.  Expectations  were  set  for  each  of  these  criteria  in  order  to  assist  in  the 
evaluation  of  the  systems.  Capability  statements  were  to  be  identified  and/or  inferred  from  the 
proposals.  They  were  mapped  to  the  criteria  and  followed  by  a  score  that  numerically  graded  each 
capability.  Tables  2  and  3  illustrate  a  sample  of  the  criteria  and  the  grading  scale  respectively. 


Table  2:  Illustration  of  the  Evaluation  Template — A  Sample 


Proposed  Budget  System  Proposal  Evaluation  Template 

Vendor 

Name: 

Requirement 

Expectation 

Capability 

Grade 

Comments 

Client/Server 

Application 

Performance^ 

Should  be  quick  and  scalable  as 
the  user  base  grows.  Average 
response  time  should  optimize 
on  speed,  bandwidth,  traffic, 
process,  etc. 

2 

As  one  Senior  System  Administrator  pointed  out,  it  is  important  to  quantify  the  attributes  sought  of  the 
system.  In  the  illustration  above,  it  would  be  meaningful,  for  example,  to  quantify  “quick”  with  “sub¬ 
second  response  time”  against  the  “performance”  attribute.  In  our  specific  case,  the  application  was 
meant  to  serve  a  small  user  base  and  we  incorporated  a  section  in  the  functional  specifications  document 
requiring  that  we  be  told  what  the  expectations  could  be  from  the  systems  proposed.  Our  Senior  Database 
Administrator  endorsed  the  capability  of  the  chosen  system  with  respect  to  its  satisfaction.  It  is  advised 
that  the  need  for  satisfaction  of  specific  system  attributes  be  appropriately  quantified  on  a  case-by-case 
basis. 
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Table  2:  Illustration  of  the  Evaluation  Template— A  Sample  (Continued) 


Compatibility 


Should  be  compatible  with  the 
current  technical  environment. 
Windows  NT/2000  or  UNIX 
Solaris  2.6  or  higher,  Oracle 
8.1.6  or  higher,  SQL  Server 
2000,  Oracle  AMG  APIs. 


Table  3:  Illustration  of  the  Grading  Scale  Used — A  Sample 

Grading  Scale  for  the  New  Budget  Solution  Proposal  Evaluation 

10  Fully  addressed  in  current  version 

8  Partially  addressed  in  current  version;  low-cost,  no-impact/low-impact 
minor  modifications  will  return  fully  desired  functionality 

7  Not  addressed  in  current  version;  low-cost,  no-impact/low-impact 
modifications  will  return  fully  desired  functionality 

6  Partially  addressed  in  current  version;  high-cost,  no-impact/low-impact 
modifications  will  return  fully  desired  functionality 


All  evaluators  were  provided  the  guidelines  for  evaluation  in  unambiguous  terms  at  the  proposal¬ 
opening  meeting.  Some  of  the  interesting  characteristics  of  this  process — as  seen  from  the 
evaluators’  comments  at  the  end  of  this  paper —  were  the  following: 

•  Evaluators  felt  respected  by  the  level  of  participation  afforded. 

•  Evaluators  were  allowed  to  evaluate  not  only  criteria  that  mapped  to  their  field  of  expertise 
but  also  other  aspects  of  the  proposal  if  they  chose. 

•  Core  technical  staff  members  voiced  happiness  to  be  involved  in  the  process. 

•  Common  understanding  of  the  capabilities  of  the  solutions  existed. 

•  Most  evaluators  turned  in  valuable  evaluation  comments. 

•  Grades  appeared  to  be  based  on  the  evaluators’  understanding  of  the  proposal. 

•  Experts  were  used  to  review  the  proposals  for  better  understanding. 

•  New  questions  were  generated  for  the  vendors’  clarifications. 


5.5  Consolidation  of  Comments  and  Grades 

All  comments  and  grades  received  from  various  individuals  were  consolidated  for  each  COTS 
vendor  that  had  submitted  a  proposal.  When  the  results  came,  they  were  quite  interesting:  there 
was  less  than  half  a  point  separating  three  of  the  four  serious  contenders,  the  last  having  been 
discarded  on  grounds  of  not  reflecting  a  comprehension  of  the  needs. 
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5.6  Reference  Calls,  Vendor  Calls,  and  Research  Group  Analysis 

The  COTS  process  did  not  lack  advice  for  the  situation.  Inferences  drawn  from  the  COTS  training 
prompted  us  to 

•  use  references  provided  by  the  vendor 

•  use  references  known  to  internal  members  developed  through  proper  networking  and  public 
relations  with  counterparts  in  the  market  place 

•  continue  interaction  with  the  vendor  on  various  issues  that  crop  up  from  time  to  time 

Over  a  dozen  references  were  called;  these  were  spread  across  the  country  and  included 
organizations  large  and  small  (universities,  FFRDCs,  and  corporate  bodies)  that  had  implemented 
one  or  another  of  the  products  in  question.  We  sought  further  validation  through  product  research 
analyses,  which  pointed  to  significant  problems  regarding  maintenance,  implementation  and 
modifiability  concerns,  as  well  as  scarcity  of  skilled  resources.  This  left  just  one  fully  capable 
COTS  solution  to  choose. 


5.7  Recommendation 

If  not  for  the  COTS-based  process  undertaken,  preparing  the  recommendation  would  have  been  a 
Herculean  task — or  so  concurred  our  team  members,  based  on  their  previous  experiences.  The 
recommendation  with  the  supporting  material  was  ready  in  quick  time  for  management’s 
consideration.  The  frequent  updates  to  management  and  the  strength  of  our  process  helped  clear  us 
for  further  negotiations  with  the  vendor  on  terms —  also  within  quick  time. 


5.8  System  Design  -  A  Perception 

To  assist  in  the  resource  allocation  efforts  with  an  aim  of  reducing  the  cost  of  implementation,  we 
embarked  on  an  effort  to  crystallize  the  issues,  perceptions  and  interactions  within  the  environment 
that  the  new  system  would  be  chartered  into  for  a  view  that  would  foster  reasoning.  Several 
references^  were  used  to  construct  the  view.  The  effort  resulted  in  an  architectural  rendering  (Figure 
1)  of  the  systems’  context  view.  The  components  seen  above  and  below  it  represent  the  ERP  and 
HRIS  systems  that  the  new  system  would  rely  upon  and  offer  guarantees  to.  The  rendering  helped  to 


^  The  following  definitions  of  software  architecture  were  used  to  provide  guidance  in  the  effort; 

•  the  structure  of  the  components  of  a  program/system,  their  interrelationships,  and  principles  and 
guidelines  governing  their  design  and  evolution  over  time  [Garlan  95]. 

•  the  structure  or  structures  of  the  system,  which  comprise  software  components,  the  externally  visible 
properties  of  those  components,  and  the  relationships  among  them.  By  externally  visible  properties,  we 
are  referring  to  those  assumptions  other  components  can  make  of  a  component,  such  as  its  provided 
services,  performance  characteristics,  fault  handling,  shared  resource  usage,  and  so  on  [Bass  98]. 

•  the  fundamental  organization  of  a  system  embodied  in  its  components,  their  relationships  to  each  other 
and  to  the  envirorunent,  and  the  principles  guiding  its  design  and  evolution  [IEEE  00]. 
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1.  map  the  SEI’s  and  CMU’s  internal  resources  with  the  roles,  responsibilities,  and  skills  that 
were  required  for  successful  implementation  of  the  solution 

2.  redefine  the  roles,  responsibilities,  and  related  skills  that  the  vendor  had  to  bring  along  to 
contribute  to  the  success  of  the  project 

3.  refine  a  proposal  that  was  heavy  on  consulting  costs,  essentially  trimming  it  by  about  a  half 
of  the  costs  originally  proposed 

4.  clarify  the  assignment  of  tasks  foreseen  through  an  evaluation  of  the  vendors’  proposal 

5.  foster  an  understanding  of  the  integration  points  that  must  be  satisfied  to  make  the 
implementation  a  success 


LEGEND  for  Figure  2,  next  page 
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Figure  2:  Context  Diagram  Representing  the  Integrated  Environment 

Note:  Different  degrees  of  shading  distinguish  different  full-fledged  systems. 
All  client  software  may  not  be  installed  for  all  users. 
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6  Value  in  Approach 


The  Manager  of  Financial  and  Administrative  Services  anticipates  an  organizational  culture 
change  regarding  future  COTS  acquisitions — ^based  on  the  positive  experiences  derived  through 
the  adoption  of  COTS  techniques  in  this  project  execution.  Let  us  capture  the  lessons  learned 
through  the  use  of  this  approach. 


6.1  Lesson  1  -  Faith  Is  Fifty  Percent  of  Healing 

If  religiously  practiced  as  they  were  by  this  team,  the  COTS  acquisition  techniques  offer  the 
promise  of  strong  gains  that  can  be  directly  experienced.  These  techniques  are  known  for  their 
robustness  in  helping  to  capture  intricacies  and  weaknesses  ahead  of  product  selection.  Simple 
examples  include 

•  the  ability  to  realize  that  one  of  the  strong  contenders  relied  on  software  services  that  were 
the  subject  of  fresh  alerts  in  terms  of  their  vulnerability  to  security  attacks 

•  knowledge  of  increased  coding  efforts  to  keep  the  system  operational  in  an  evolving  IT 
environment  with  respect  to  another  of  the  contenders 

•  the  feeling  of  being  in  control  most  of  the  time 

The  demands  of  the  process  were  so  high  that,  at  times,  a  tendency  to  compromise  on  the 
discipline  crept  in.  But  the  hope  of  success  in  the  final  analysis  urged  us  to  persist  on  the  chosen 
path.  All  round,  confidence  was  high.  Management  and  stakeholders  reposed  great  faith  in  the 
team  that  had  reposed  great  faith  in  the  process.  The  project  earned  respect  due  to  the  approach. 
The  vendors  cooperated  through  the  entire  process. 


6.2  Lesson  2  -  Knowledge  Is  Power 

The  process  fostered  a  deep  understanding  of  the  goals  and  the  means  to  reach  them.  It  helped 
field  the  right  questions  to  the  vendors  and  stakeholders  at  the  right  time  (refer  to  Section  5).  It 
promoted  the  reasoning  of  every  single  aspect  that  was  relevant  or  critical  to  the  success  of  the 
project.  The  difficult  side  was  finding  the  right  mix  of  stakeholders  and  managing  their 
aspirations.  The  knowledge  of  the  team  responsible  for  the  project  should  be  equal  to  or  greater 
than  the  sum  of  the  knowledge  of  the  stakeholders  as  the  project  progresses.  This  was  the  only 
way  power  balance  could  be  maintained. 
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6.3  Lesson  3  -  We  Reap  What  We  Sow 

It  is  important  to  support  the  evaluation  process  with  the  right  mix  of  resources,  a  proper 
schedule,  and  a  concerted  effort.  Some  gains  we  experienced  are  represented  in  the  answers  to  the 
following  commonly  asked  questions. 

Question:  What  does  one  stand  to  gain? 

Answer:  The  following  listed  benefits  are  some  that  we  experienced.  They  are  derived  from  the 
efforts  mentioned  in  the  sections  referred  to  in  parentheses. 

•  ability  to  make  well-informed  decisions  (6.1,  6.2,  6.3, 6.5) 

•  ability  to  take  calculated  risks  through  appropriate  tradeoffs  (3.3,  6.5) 

•  thorough  comprehension  of  the  known  (3.3) 

•  preparedness  to  seek  out  and  face  the  unknown  (3.4,  3.5,  all  of  section  4) 

•  upfront  knowledge  of  all  the  main  and  ancillary  components  and  services  needed  as  well  as 
their  costs  to  make  the  system  fully  operational  (6.3) 

•  anticipated  reduction  in  the  day-after-the-fair  surprises  (6.1,  6.2,  6.3,  6.5) 

•  wisdom  of  a  well-analyzed  investment  (4.3,  6.4) 

We  expect  a  shorter  time  to  market  since  all  the  players,  functional  and  technical,  together  are 
aware  of  all  the  building  blocks  needed  to  make  this  system  operational  in  our  environment.  We 
only  need  to  bring  together  the  vendor’s  resources — both  consultants  and  software — and  the 
internal  resources — people  and  hardware — to  get  started  with  the  implementation. 


Question;  Can  the  return  be  quantified? 

Answer:  While  the  return  can  be  quantified  in  terms  of  the  increased  productivity  and  reduced 
risk  anticipated,  it  remains  to  be  seen  what  the  facts  are  based  on  a  post-implementation  audit. 
However,  we  can  conclude  the  following: 

The  risk  of  making  a  poor  investment  is  reduced;  there  should  exist  a  correlation  between 
the  amount  of  effort  spent  and  the  business  value  expected  to  be  gained  by  following  COTS 
product  acquisition  techniques  (6.5). 

Results  are  the  effect  of  effort;  the  more  the  effort,  the  better  the  results;  the  depth  of  evaluation 
determines  the  strength  of  the  foundation;  having  said  this,  we  shall  wait  and  watch  what  the 
results  are  considering  that  we  have  put  in  a  fairly  large  effort. 
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7  Conclusion  -  Experience 


Adoption  of  the  process  and  adherence  to  the  techniques  in  their  true  spirit  was  a  difficult  grind. 
Validating  all  that  was  said  and  heard  required  amazing  degrees  of  commitment.  The  fact  that  a 
greater  part  of  the  validation  was  done  through  external  resources  made  it  even  harder.  It  should 
be  mentioned,  though,  that  almost  all  of  our  consulted  resources  were  extremely  kind  in  getting 
us  the  necessary  input  on  the  way  to  our  decision  and  recommendation.  The  need  for  such 
validation  was  even  higher  since  there  is  an  internal  process  referenced  by  our  DoD  contract  that 
keeps  us  functional,  which  requires  a  Request  for  Consent  before  any  funding  is  committed.  The 
COTS  acquisition  techniques  did  make  this  process  a  bit  easier  since  we  were  ready  with  all  the 
necessary  documentation  for  the  purpose. 

As  for  the  team,  there  was  a  very  high  degree  of  cooperation  in  getting  tasks  executed.  There  was 
a  high  degree  of  patience  with  the  hope  of  getting  something  good  out  of  the  effort.  It  certainly 
was  a  tough  call  for  team  members  to  make  time  available  from  their  busy  schedules  to 
accommodate  the  needs  of  the  process  that  we  adopted.  It  is  also  important  to  note  that  the 
managers  of  the  project  were  committed  to  devoting  the  right  amount  of  time  for  the  work  that 
needed  to  be  done.  We  had  a  scheduled  weekly  meeting  until  the  point  in  time  that  the  evaluations 
had  to  begin.  At  every  meeting  there  were  action  items  clearly  specified.  Necessary  tasks  were 
accomplished  and  formed  the  basis  of  discussions  in  the  ensuing  meeting.  These  meeting  were 
used  to  deliver,  reason  out,  validate,  and  plan  all  the  building  blocks  needed  to  begin  the 
implementation. 

The  Chief  Financial  Officer  was  updated  on  the  status  of  the  project  on  a  weekly  basis  during  the 
one-on-one  meetings  with  the  project  manager.  Corrective  action  was  prompt  in  being  executed. 
Professional  guidance  was  sought  and  applied  where  necessary  with  no  delays.  Every  individual 
on  the  team  and  within  the  stakeholder  groups  played  a  key  role  in  delivering  the  necessary  input 
to  bring  the  evaluation  process  to  a  successful  close. 


7.1  Comments  from  the  Team  Members  on  Their  Experience 

Let  us  hear  what  the  Manager  of  Financial  Analysis,  a  key  member  on  the  team,  had  to  say: 

"Before  participating  in  the  COTS  evaluation  process,  I  was  very  skeptical  that  a  COTS  product 
could  solve  our  problems.  But,  by  being  involved  in  this  process,  I  am  satisfied  that  we  have 
found  our  best  solution.  ” 

A  Business  Analyst  at  the  SEI  had  the  following  to  say: 
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''The  COTS  process  enabled  the  Budget  Team  to  discriminate  between  the  requirements  that  were 
absolutely  necessary  and  those  requirements  that  we  could  live  without.  This  was  in  my  opinion 
the  most  challenging  part  of  the  entire  process.  Had  it  not  been  for  the  training  and  guidance 
provided  by  the  COTS  folks,  the  determination  of  the  necessary  attributes  and  those  that  are  not 
necessary  would  not  have  been  completed  with  the  same  degree  of  accuracy.  ** 

The  COTS  Program  Specialists  rendered  the  following  opinion: 

"The  team  was  totally  inexperienced  in  this  kind  of  selection,  naive  about  some  aspects — in  other 
words,  completely  typical  of  a  team  facing  such  a  challenge  for  the  first  time.  Their  success  is,  in 
our  opinion,  due  to  their  ability  to  understand  and  their  willingness  to  embrace  the  principles  we 
introduced  into  the  discussions.  ” 

A  Senior  System  Administrator  had  the  following  to  say: 

"...some  of  the  evaluation  parameters  were  not  quantified.  Specifically,  it  indicates  that 
(paraphrased)  performance  should  be  quick  and  optimized  for  bandwidth....  Which  begs  the 
question,  'how fast  is  quick?'  Numbers  are  extremely  important...  ”  And  he  went  on  to  add,  when 
asked  about  the  pre-proposal  conference,  "It  was  a  good  exercise. " 


7.2  Expectations 

The  process  appears  to  support  a  shorter  final  leg  of  the  project.  Confidence  in  this  assertion 
stems  from  the  fact  that  all  players  involved  in  its  implementation  are  well  educated  and  informed 
about  the  project  goals,  the  process  goals,  the  roles,  the  responsibilities,  and  the  accountability. 
Though  the  final  outcome  is  months  away,  it  seems  that  the  project  is  poised  for  a  smooth  journey 
ahead.  Look  out  for  the  facts  in  future  papers  in  the  Preacher’s  Practice  series.  The  series  is 
committed  to  reinforcing  the  software  engineering  discipline  with  facts  from  the  home  turf,  the 
Software  Engineering  Institute,  Pittsburgh,  PA  15213,  U.S.A. 
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Acronyms 


ACIS 

Administrative  Computing  Information  Services 

BDI 

Budget  Development  Integrator 

CBS 

COTS-based  systems 

CFO 

Chief  Financial  Officer 

CMU 

Carnegie  Mellon  University 

COTS 

commercial  Off-the-shelf 

ERP 

Enterprise  Resource  Planning 

FABS 

Financial  and  Business  Services 

FFRDC 

federally  funded  research  and  development  center 

GM 

Grants  Management  (An  Oracle  ERP  Module) 

GL 

General  Ledger  (An  Oracle  ERP  Module) 

HRIS 

Human  Resource  Information  System 

rr 

Information  Technology 

RFC 

request  for  consent 

RFP 

request  for  proposal 

SEI 

Software  Engineering  Institute 
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